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ABSTRACT 


A  procedure  for  the  design  of  a  controller  to  optimize  a  certain  class  of 
pulse  width  modulated  systems  is  presented.  The  process  to  be  controlled  is 
time- invariant,  of  arbitrary  order  and  excited  by  a  sequence  of  pulses  gen¬ 
erated  from  information  available  at  arbitrary  sampling  instants.  Input  infor¬ 
mation  to  the  system  is  quite  general  and  includes  random  and  deterministic 
phenomena.  Identification  of  the  plant  is  accomplished  using  state  variable 
notation  and  linear  estimation  techniques.  Prediction  of  the  future  plant 
behavior  is  also  performed  with  these  techniques  and  the  controller  is  designed 
to  optimize  the  predicted  plant  performance  by  minimizing  a  measure  of  the 
future  system  errors.  The  mechanized  optimal  control  law  or  controller  program 
develops  the  pulse  to  be  applied  at  any  sampling  instant  by  specifying  the 
pulse  width  and  associated  sign.  The  controller  is  not  adaptive  in  the  sense 
of  redesign  occuring  as  new  information  becomes  available  from  the  estimator. 

It  does,  however,  accomplish  the  goal  of  optimization  by  deciding  the  form 
of  the  pulse  width  actuating  signal  utilizing  the  future  errors  of  the  system. 

The  general  results  are  applied  to  several  examples  through  digital  computer 
simulation.  The  optimal  pulse  width  controller  is  shown  to  produce  far  better 
performance  than  normal  pulse  width  control  which  utilizes  present  errors  to 
develop  the  actuating  signal  to  be  applied  at  that  same  instant  and  for  some 
time  in  the  future.  The  results  of  considering  a  family  of  step  function  in¬ 
puts  to  the  system  are  shown  for  a  third  and  fourth  order  plant. 
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1.  INTRODUCTION 


1.1  Pulse  Width  Control 

Pulse  width  modulated  control  systems  have  been  In  existence  for  more  than 
65  years^  and  only  recently^  activity  and  Interest  have  Increased  in  this  area. 
The  present  frequent  use  of  pulse  width  control  has  developed  in  spite  of  the 
almost  complete  lack  of  elegant  mathematical  tools  for  analysis  and  synthesis 
since  the  description  of  these  systems  requires  nonlinear  differential  or  dif¬ 
ference  equations.  This  seems  to  Indicate  the  possibility  that  this  type  of 
control  is  so  effective  and  natural  in  a  large  class  of  control  problems  that 
defiance  of  analysis  is  not  sufficient  Justification  to  avoid  its  use. 

Pulse  width  control  is  a  generalized  form  of  relay  or  on-off  control  which 
provides  a  finer ^  more  precise  response.  The  advantages  arise  mainly  from  the 
ability  to  regulate  the  steady  state  ripple  oscillation  frequency^  to  obtain 
Improved  accuracy  due  to  elimination  of  dead  zone,  and  to  include  possible  time 
sharing  of  the  control  computer. 

Pulse  width  modulated  control  has  direct  application  to  satellite  and 

space  vehicle  attitude  control.  In  many  cases  this  situation  requires  power 

to  be  modulated  in  an  on-off  fashion  where  a  control  computer  must  be  time 

shared  leaving  very  little  choice  other  than  to  use  pulse  width  control.  Recent 

experimental  efforts  in  this  area  have  shown  that  time  dependent  switching 

techniques  provide  precision  attitude  control  with  low  thrust  vapor  Jets  achiev- 
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Ing  results  not  possible  with  conventional  on-off  methods  *  . 

Further  investigation  of  pulse  width  control  is  Justifiable  on  the  basis 
of  attitude  control  applications  alone,  but  let  us  consider  for  a  mcxnent,  another 
system  utilizing  a  more  general  form  of  pulsed  control. 

Han  is  the  ultimate  control  system.  Many  physical  devices  have  been  modeled 

after  a  particular  human  function,  and  today  we  find  an  ever  increasing  effort 

toward  the  understanding  of  biological  functions  so  that  related  systems  may  be 

improved  and  extended.  The  method  of  information  transmission  in  the  human 

being  is  a  combination  of  pulse  width  and  pulse  repetition  modulation.  We  find 

4 

cardiac  pulsatory  phenomena  remarkable  in  all  senses  ,  particularly  in  accuracy 

and  reliability  of  control,  since  it  is  literally  vital.  More  incredible  is 

.  5.6 

the  completely  Integrated  pulse  communication  network  in  the  nervous  system  ’  . 
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The  sensory  receptors  provide  the  pulsed  form  oi  physical  stimuli  that 
our  system  achieving  extreme  sensitivities  (e,g.  the  retina  oi  the  eye) .  Fr ;m 
these  considerations  it  might  be  concluded  that  pulse  modulation  would  b«  the 
ideal  method  of  information  transmission  for  many  control  purposes,,  In  any  case^ 
future  investigation  of  this  type  and  simpler  types  of  pulsed  control  is  cer¬ 
tainly  justifiable. 

Several  authors,  R.F.  Nease^,  R.E.  Andeen^,  T.T,  Kadcta^,  E.  Polak^^" ^ ^ ^ ? 

T.  Nishimura  and  E.I.  Jury^"^,  I.V.  Pyshkin^^’ F.R,  Delfeld  and  G,J,  Murphy^^, 

18  19 

S.C.  Gupta  and  E.I.  Jury  ,  S.C.  Gupta  ,  have  published  efforts  in  limit  cycles, 
finite  duration  processes,  analytical  techniques  and  opt imizat icn .  Ihe  techniques 
for  optimization  have  been  limited  to  second  order  regulator  systems  to  date. 

The  purpose  of  this  work  is  to  develop  an  optimization  technique  applicable  to 
controlled  processes  of  any  order  when  the  system  is  subjected  to  a  broad  class 
of  possible  input  phenomena, 

1.2  Statement  of  the  Problem 

The  system  to  be  investigated  in  this  report  is  assumed  quite  general  in 
form.  Restrictions  on  the  nature  of  the  system  which  would  degenerate  useful¬ 
ness  of  the  optimization  technique  to  only  academic  problems  are  avoided. 

Practical  considerations  have  taken  precedence  in  the  formulation  of  this  problem 
where  ever  possible. 

A  time- invar iant  linear  system  called  the  controlled  system  or  plant  is 
assumed  excited  by  a  pulse  width  modulated  control  signal.  A  feedback  centre  Her 
is  to  be  designed  to  provide  the  best  control  or  actuating  signal  utilizing 
information  about  the  state  of  the  system  at  only  arbitrary  sample  inst-ints. 

The  information  about  the  state  of  the  system  is  constrained  in  that  only  noisy 
measurements  are  available  for  estimation  purposes.  The  system  is  to  be 
optimum  for  this  pulse  width  modulated  control  signal  where  optimum  is  used 
in  the  sense  of  minimizing  a  measure  of  system  error. 

The  mechanized  control  law  will  constitute  the  design  of  the  optimal  con¬ 
troller  and  the  solution  to  the  problem.  The  controller  is  not  to  be  adaptive 
in  the  sense  of  redesign  occuring  as  new  information  becomes  available  about 
the  state  of  the  system.  It  is  to  provide  a  program  of  logical  steps  that 
will  be  executed  during  each  sampling  interval  determining  the  exact  fer- 
the  pulsed  actuating  signal  by  specifying  the  pulse  width  and  associai 
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The  command  Inputs  to  the  system  are  quite  general  in  form  in  that  they 
include  both  random  and  deterministic  phenomena. 

A  block  diagram  of  the  system  under  investigation  in  this  report  is  shown 
in  Figure  1. 

I. 3  Optimal  Control  Law 

The  optimization  of  a  class  of  pulse  width  modulated  systems  with  linear 

20 

plants  was  first  considered  by  W.L.  Nelson  .  This  class  consisted  of  regulator 

systems  and  is  a  problem  of  minimal  time  control.  Nelson's  approach  follows  a 

general  technique  for  the  minimal  time  control  of  pulse  amplitude  modulated  sys- 

21 

terns  with  saturation  as  presented  by  R.E.  Kalman  .  This  procedure  consists  of 
dividing  the  state  space  into  regions  for  which  the  system  may  be  taken  to  the 
desired  state  in  a  minimal  number  of  sampling  periods,  establishing  a  canonical 
vector  representation  for  Initial  states  in  the  respective  regions.  The  ap¬ 
proach  is  limited  to  plants  of  second  order  due  to  the  conceptual  difficulty  of 
determining  the  optimal  control  regions  in  higher  dimensional  state  spaces. 

Nelson  did  not^  however,  present  a  procedure  for  constructing  canonical 

11  12  13 

vector  representations  for  arbitrary  initial  states.  E.  Polak  '  '  suc¬ 

ceeded  in  accomplishing  this  following  a  method  suggested  by  C.A.  Desoer  and 
22 

J.  Wing  on  minimal  time  control  of  pulse  amplitude  modulated  systems  with 
saturation.  Polak  was  able  to  solve  the  regulator  problem  for  some  second  order 
linear  and  nonlinear  plants.  The  limitation  to  second  order  is  again  prominent 
due  to  conceptual  difficulty  in  higher  dimensional  state  spaces. 

The  previous  efforts  have  been  based  on  the  approaches  used  to  optimize 
pulse  amplitude  modulated  systems.  The  extension  of  techniques  appllcablo  to 
pulse  amplitude  systems  may  itself  be  a  fundamental  limitation  and  thus  not 
worthy  of  further  investigation.  Considering  pulse  width  control  as  a  funda¬ 
mentally  different  type  of  control  with  its  own  intrinsic  properties,  an  ap¬ 
proach  to  the  problem  may  be  found.  Variational  techniques  have  been  attempted 
23 

with  little  success  .  Combinatorial  techniques  as  previously  applied  seem 
to  be  the  only  other  alternative.  Dynamic  programming  as  an  organized  logical 
scheme  is  directly  related  to  the  sampled  system  and  seems  to  be  the  most 
likely  candidate  here.  Dynamic  programming  then,  will  provide  the  logical 
framework  for  the  optimization  scheme  where  the  pulse  width  modulated  control 


Figure  1.  Block  Diagram  of  the  System 


signal  will  be  treated  as  a  part  of  the  system  constraint  rather  than  as  a 
separate  constraint. 
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The  previous  contributions  in  the  application  of  dynamic  programming 

have  been  formulated  for  time  varying  plants.  The  first  such  application  to 

24 

a  time  varying  plant  was  presented  by  R.E.  Kalman  and  R.W.  Koepcke  .  Later^ 

25  26 

T.L.  Gunckel  and,  independently,  P.D.  Joseph  and  J.T,  Tou  ,  reported 

results  in  the  control  of  linear  processes  subject  to  multiplicative  as  well 

as  additive  random  effects.  A  very  practical  extension  to  Include  random 

multiplicative  phenomena  which  are  correlated  both  in  time  and  with  each  other, 

and  not  restricted  to  independence  from  Instant  to  instant,  has  been  presented 

by  C.  Pottle^^. 

The  system  under  study  here  is  not  assumed  time  varying  for  simpliiicatlon, 
but  is  approached  in  exactly  the  same  fashion  as  mentioned  above.  The  additional 
constraint  of  pulse  width  modulation  is  included  and  the  Information  about  the 
state  at  sampling  instants  is  also  constrained.  Minimization  of  a  measure  of 
predicted  system  errors  on  a  step  by  step  basis  provides  the  final  necessary 
ingredient  for  the  solution  to  the  optimization  problem. 

Estimation  of  the  state  and  prediction  of  the  future  system  error  is 

accomplished  by  linear  estimation  techniques  in  state  vector  notation  as  pre- 

28  29 

seated  by  R.E.  Kalman  * 
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2.  DESCRIPTION  OF  THE  SYSTEM 


2.1  General 

A  mathematical  description  of  the  system  is  necessary.  We  must  decide 
on  a  general  model  which  is  sufficiently  complex  to  represent  the  behavior  of 
a  large  class  of  physical  processes  with  a  high  degree  of  accuracy.  The  model 
must  also  be  simple  so  that  useful  results  may  be  obtained  easily.  Any  model 
under  consideration  is  also  required  to  represent  linear  plants  subject  to 
pulse  width  information. 

The  representation  of  a  pulse  using  transform  techniques  where  the  width 
is  a  complex  function  of  system  responses  seems  entirely  inadequate.  Thus,  a 
time  domain  representation  will  be  used. 

2 . 2  State  Vector  Representation 

A  general  model  which  can  represent  a  system  with  finitely  many  degrees  of 

freedom  is  one  using  the  "state  space"  concept  introduced  by  Kalman  and 
30 

Bertram  .  The  state  of  the  system  at  any  instant  of  time  is  represented  by 
a  vector  whose  components  are  called  the  "state  variables."  This  vector  defines 
a  point  in  the  state  space.  The  dimension  of  the  state  vector  is  the  smallest 
possibly  to  completely  describe  the  behavior  of  the  system  at  that  point  in  time. 

For  a  general  time  invariant  plant,  its  dynamic  behavior  is  assumed  to  be 
adequately  approximated  by  the  following  vector  equation. 


where 


x(t)  =  A  x(t)  +  B  u(t)  (2.1) 

c(t)  =  x(t) 

x(t)  is  an  (mxl)  state  vector 

c(t)  is  a  (pxl)  output  vector 

u(t)  is  a  (qxl)  input  vector 

A  is  an  (mxm)  transition  matrix 

B  is  an  (mxq)  distribution  matrix 

M  is  a  (pxq)  output  matrix 


If  the  system  is  time  invariant,  the  transition,  distribution  and  output 
matrices  have  elements  not  dependent  on  time. 
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The  actuating  or  control  signal  is  u(t)  and  will  have  a  form  dictated  by 
the  constraints  Imposed  by  the  problem.  For  the  particular  problem  at  hand, 
that  of  pulse  width  modulated  systems,  the  control  signal  must  then  satisfy 
these  requirements.  The  1th  component  of  u(t)  Is 

00 

UfCt)  =  Y  [“i<t-^)  -  ^nl>Xl 

n=0 

where  u, (t)  Is  the  unit  step  function  and  P  .  Is  the  pulse  width  of  the  nth 
1  nl 

Interval  for  the  1th  channel  and  £  .  Is  the  associated  sign  (*1).  Thus,  we  see 

nl 

that  the  pulsed  control  signal  Is  described  by  the  sequence  of  pairs  of  vectors 
(P^,€^) .  The  solution  to  the  optimization  problem  Is  the  generation  of  a  se¬ 
quence  of  couples  determined  In  such  a  way  that  the  performance  of  the 

system  satisfies  the  condition  for  optimality. 

For  the  control  vector  u(t),  we  have 


where 


“l ‘nl 
"l<'-*„-  ‘n. 


P  Is  a  (qxl)  pulse  width  vector 
n 

€  Is  a  (qxl)  sign  vector 
n 


Defining  a  unit  step  function  matrix  will  simplify  the  description  of 
the  control  vector  In  general  form.  Let 


Uj(t)  =  I  u(t) 


where  I  Is  a  (qxq)  unit  matrix. 
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I  = 


10  0 
0  1  . . 


0 

0 


0  0 


And,  let  Uj^(a),  where  a  is  a  (qxl)  vector,  be  defined  as 


U^(a)  = 


Uj(aj^)  0  0 

0  u^Ca^) 


0 

0 


Then,  the  control  vector  may  be  written 


u(t)  =  y  [n  (t-t  )  -  U, (t-t  -  p  )  1  €  (2.2) 

^L1  ^  1  a  njn 


n=0 


Using  this  notation,  we  may  now  determine  the  discrete  form  of  the  state 
vector  representation  for  the  system  at  the  sampling  instants.  The  solution 
for  the  state  vector  in  equation  (2.1)  is 


i(t)  =  </J*(t,t  )x(t  )  +  f  0*(t,\)Bu(\)d\  t  (2,3) 

n  n  J  ^  '  n 

n 

where  (f^(t,t  )  =  e^^^  and  is  called  the  characteristic  matrix, 
n 

Using  the  expression  developed  for  the  control  signal  u(t),  equation  (2.3) 
becomes 


iCt)  =  <#>*(t,t  )x(t  )  + 
n  n 


for  t  <  t  <  t  -  . 
n  —  —  n+1 


f  4>(t,\)Bd\ 
•  tn 


t 


I 

I 

I 

I 

I 


To  •iBpllfy  notation  define 


where  H  =  t  =  l,2,..q  J  =  and  b  Is  a  (qxl)  vector  =  (b^^) 

k  a  1,2, . .q. 

The  expression  for  x(t)  may  be  written  now  as 


”  t  t 

x(t)  =  <^i*(t,t^)jc(t^)  +  <^i*(t,\)Bd\  -  j  ‘^^t,\.)BUj^(\-tj^-/j^)d\ 


(2.4) 


For  t^  +  5^5  ^n+1  ^  ~  Equation  (2.4)  may  be  simplified 

by  combining  the  two  Integrals. 

V^nh 

i(t)  =  </)(*(t,t^)x(t^)  +  J  ^(t,\)  B  d\  (2.5) 


1  —  1^2^ 


Changing  the  variable  of  integration  in  (2.5) 


=  ^(t,t  )x(t„)  +  f  B  dp  €  (2.6) 

n  n  J  ^  n 


n  1 

=  <^>*(t,t  )x(t  )  +  <#ll“(t,t  )  J  e"^P  B  dp  (2.7) 

n  n  n  V  A  ii 


for  1  s  1,2, .. .q 


+  (Pj.  <  t  <  t  . 

n  n  1  —  —  n+1 


i 
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For  the  response  at  the  sampling  Instants,  t  =  t  ,  and  equation  (2.7) 

n+1 


becomes 


*'Vl>  “  ‘n  “.8) 


Letting  x(t  )  =  x  and  equation  (2.8)  becomes 

Ij  Jj  I  '  n' 


n+l»  n'  ^n' 


X  =  0*  X  +  <^  I  B  dX.  e 

n+1  ^n  n  ^n  J_  n 


(2.9) 


The  controlled  system  is  thus  adequately  approximated  at  the  sampling 
Instants  by  the  vector  equations 


X  =  0*  X  +  A  t 
n+1  n  n  n  n 


c 

n 


M*  X 

n  n 


(2.10) 


where  (A  )  ^ 

n  1 

f  e"^  B  d\  i=l,2,...q  (2.11) 

n  n  Jq 

The  equations  (2.10)  describe  what  will  be  called  the  plant  process  at  the 
sampling  instants.  The  state  of  the  system  at  the  sampling  Instants  is  con¬ 
trolled  by  the  choice  of  the  adjustable  parameters  in  the  control  signal  which 

are  the  pulse  widths  P  and  signs  €  .  The  control  law  will  provide  the  tech- 

n  n 

nique  by  which  this  pair  (P  ,€  )  may  be  determined  in  the  desired  manner. 

n  n 

2.3  The  Input  Process 

Kalman  has  Introduced  the  notion  of  a  reference  input  vector  r(t  )  which 

n 

can  be  regarded  as  being  generated  as  the  output  of  a  linear  dynamic  system. 

Following  this  idea,  the  input  or  reference  signal  r(t  )  is  assumed  to  be  defined 

n 

by  an  input  process.  That  is,  the  reference  input  vector  representing  the 
desired  value  of  the  plant  output  vector  is  adequately  approximated,  at  the 
sampling  Instants,  by  the  following  vector  difference  equation. 


II 


wh«re 


n+1 


W  +  A"  S 
n  n  n  n 


r  a  M  w 
n  n  n 


(2.12) 


w  is  a  (vxl)  state  vector 
n 

Is  a  (pxl)  reference  input  vector 

s^  Is  an  (hxl)  process  input  vector 

0*  is  a  (vxv)  transition  matrix 
w 

is  a  (vxh)  distribution  matrix 
is  a  (pxv)  process  output  matrix 


The  disMnsionality  of  the  reference  input  vector  r  has  been  assumed  the 

ti 

same  as  that  of  the  output  vector  c^.  The  components  of  the  process  input 
vector  are  assumed  to  be  random  variables  with  zero  mean  and  to  be  statistically 
independent  from  one  sampling  instant  to  the  next  and  independent  of  the  transi¬ 
tion  and  distribution  matrices.  Thus 


[0  n  9^  m 

E(8  )  =  0  E(s  s  )  =4  (2.13) 

n  n  m  I  . 

n  =  m 

Note  that  this  model  is  sufficiently  general  in  that  it  includes 

a)  The  regulator  problem  when  r  =0  for  all  n 

n 

w 

b)  The  deterministic  problem  when  A  =  0  for  all  n 

n 

c)  The  random  input  problem  when  r^  s  0 

Also,  a  combination  of  these  classes  of  inputs  can  be  accommodated  by  the  model 
using  suitably  defined  transition  and  distribution  matrices. 

The  assumed  input  vector  r(t)  may  or  may  not  belong  or  be  a  subset  of  the 
actual  command  input  vector  r'(t).  If  it  is  a  subset,  the  system  will  be  opti¬ 
mum  with  respect  to  r^;  if  not,  only  an  approximation  will  be  obtained,  and  an 
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approximation  at  the  sampling  instants.  The  designer  must  decide  on  a  class 
to  which  his  system  is  to  be  optimum,  and  then  determine  the  necessary 
matrices  since  the  controller  will  depend  on  these  quantities. 

The  requirement  that  the  reference  inputs  be  describable  as  the  output 
of  some  process  limits  the  class  of  possible  Inputs,  but  this  restriction  is 
not  too  severe  for  our  purpose. 


2.4  The  System  and  Information  Constraints 

22 

Following  the  suggestion  of  Kalman  and  Koepke  ,  the  equations  for  the 
plant  process  and  the  input  process  are  combined  into  a  single  equation.  This 
is  accomplished  as  follows; 


€ 

n 


(2.14) 


Defining 


z  = 

X 

n 

Y  = 

c 

r 

M 

1 - 

o 

V 

1 _ 

n 

1 

9 

1 _ 

n 

r 

n 

n 

1 

o 

,9^ 

4^^  0 

n 

A  = 

0 

A  = 

A 

n 

1 

o 

9-^e 
1 _ 

n 

1 _ 

n 

0 

we  may  write  (2.14)  as 


z 


n+1 


+  A_ 


s  + 
n 


A 

n 


€ 

n 


y 

n 


M 


n 


z 

n 


(2.15) 
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Where 


z 

n 

is 

an 

Y 

n 

is 

a 

€ 

n 

is 

a 

s 

n 

is 

an 

t 

is 

an 

is 

an 
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n 

is 

an 

H 

is 

a 

n 


(m+v  X  1)  state  vector 

% 

(3p  X  1)  output  vector 
(q  X  1)  sign  vector 
(h  X  1)  process  input  vector 
(m-i-v  X  m+v)  transition  matrix 
(m+v  X  h)  distribution  matrix 
(m+v  X  q)  distribution  matrix 
(2p  X  m+v)  output  matrix 


The  complete  set  of  equations  (2.15)  which  include  the  plant  process  and 
the  input  process  will  be  called  the  system  constraint. 

The  remaining  constraint  is  concerned  with  the  amount  of  information 
available  about  the  state  of  the  system  at  any  particular  sampling  Instant. 

If  the  special  case  of  exact  measurement  of  the  state  exists,  then  we  have  no 
rpstriction  on  information  and  no  need  to  consider  the  limitation.  The  usual 
case,  however,  is  the  one  where  only  certain  measurements  are  available  and 
these  measurements  together  with  past  history  constitute  the  data  available 
for  estimation  of  the  state.  Since  the  existing  data  in  this  case  is  limited 
by  additive  noise,  for  example,  we  speak  of  the  information  constraint. 
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3.  OPTIMUM  PLANT  CONTROL 


3 . 1  General 

We  are  Interested  in  finding  an  "optimum"  control  law  for  the  pulse  width 
modulated  system  described  previously.  The  sense  of  the  word  "optimum"  is 
defined  by  the  introduction  of  a  performance  index  which  provides  a  measure  of 
the  system  behavior.  The  method  of  dynamic  programming  is  then  used  to  de¬ 
rive  a  control  law  for  the  processes  of  the  type  introduced  in  Chapter  2. 
Actually,  the  method  is  applicable  to  many  cases  of  the  general  model  with 
varied  constraints,  but  here  we  will  examine  only  one  particular  class  of  prob¬ 
lems,  namely  that  of  pulse  width  modulated  systems. 

3.2  The  Performance  Index 

An  optimum  control  system  is  characterized  by  a  performance  index  which  is 

a  function  of  the  system  variables  and  parameters.  For  extremal  values  of  the 

scaler  function,  the  system  is  said  to  be  optimum  in  this  sense,  but  obviously, 

its  optimality  is  subject  to  the  performance  index  chosen.  This  choice  of  a 

performance  index  is  thus,  perhaps  the  most  important  decision  to  be  made. 

Unfortunately,  there  is  really  no  way  a  designer  may  specify  what  he  wants  in 

the  performance  of  a  particular  system  so  that  it  is  optimum  in  any  absolute 
31 

sense  .  We  are  thus  faced  with  choosing  a  performance  index  which  is  con¬ 
venient  to  work  with  mathematically  and  which  also  corresponds  to  some  reason¬ 
able  definition  of  "desirable  behavior.” 


The  measure  of  system  performance  which  seems  suitable  in  general  would 
be  the  weighted  error  of  the  system.  This  error  in  the  system  is  the  difference 
between  the  actual  plant  output  and  the  desired  output.  It  is  assumed  that  a 
quadratic  combination  of  the  error  vector  will  be  a  suitable  measure  of  perfor¬ 
mance  . 


E 

n 


(c  -  r  Q’  (c 
n  n  n  n 


r  ) 
n 


(3.1) 


The  matrix  Q'  is  taken  to  be  symmetric  and  positive  definite  and  simply  cor- 
n 

responds  to  the  desired  weighting  of  the  errors  in  the  system.  Thus,  the  single 

number  E  represents  the  performance  of  the  system  at  the  nth  Instant  of  time, 
n 

The  measure  of  performance  may  be  rewritten  in  terms  of  the  state  vector 
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Since 


then 


or 


Defining  a  new  weighting  matrix  Q 

n 


we  have 


E 

n 


-T 

z*  Q  z 
n  n  n 


(3.2) 


(3.3) 


(3.4) 


for  the  measure  of  system  performance. 

For  operation  of  the  system  involving  N  steps,  a  reasonable  performance 

index  is  the  sum  of  the  individual  performance  measures  E^. 

Define  the  performance  index  as  J„ 

N-n+1 


N 

-  I  \ 

l=n 

The  performance  index  is  thus  the  sum  of  the  weighted  mean  square  errors  at 
the  sampling  Instants. 

3.3  Criterion  for  Optimal  Performance 

To  obtain  optimum  pulse  width  control,  we  will  minimize  the  performance 
index  by  generating  the  sequence  of  both  the  width  and  sign  of  the  pulse  at 
each  sampling  instant.  The  final  sequence  of  pairs  and  the  control  law  to  ob¬ 
tain  them  will  constitute  the  solution  to  the  optimization  problem.  Optimum  is 
used  here  in  the  senre  of  minimizing  this  quadratic  function  of  system  error. 
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The  precise  statement  of  the  optimization  problem  is  thus  the  minimization  of 
the  performance  index  over  the  class  of  acceptable  pairs  of  pulse  width  and  sign, 
subject  to  system  and  information  constraints. 

That  is 

Minimize  ^ 

(P  ,€  )  N-n+1  ^3  gj 

n'  n 


Note,  however,  that  the  above  statement  is  restricted  to  only  deterministic  con¬ 
ditions.  In  the  more  general  case,  random  variations  in  the  system  preclude  the 
possibility  of  finding  an  input  which  will  be  optimum  in  every  case.  In  other 
words,  the  performance  index  is  also  a  random  variable.  The  conditional  expec¬ 
tation  of  the  performance  index  will  then  be  minimized.  Let  D  represent  the 

n 

Information  available  for  the  determination  of  the  state  of  the  system  at  time 

t  =  t  ,  The  optimization  problem  may  be  stated  as  follows: 
n 


where 


Minimize  E  /  D^) 

N 

■'n-n+l  ■  1  "’’i  'l  'S 


l:^n 


subject  to  the  system  constraint 


z 

n+ 


1 


= 


n  n 


+  A. 


A  £ 
n  n 


Y  =  M  z 
n  n  n 

and  the  information  constraint 


D  =  (y'  ,  Y'  . . ..Y*  } 

n  L  n  '  n-1*  o  J 

D  is  the  set  of  measurements  at  time  t  =  t  . 
n  n-  i 

available  for  estimation  of  the  state. 


A  loss  function  is  defined  as  follows; 

Minimum 

Iv,  ,  )'^  E  (J  ,  /  D  ) 

N-n+1  n’  n  J  N-n+1  n 


(3.8) 
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In  order  to  clarify  the  notation  used  above,  consider  a  siiiq>le  example  of 
an  N  stage  process. 

Let  N  =  the  total  number  of  stages  in  the  process 

n  s  the  number  of  the  stage  of  interest. 


When  at  stage  n,  the  number  of  remaining  stages  which  constitutes  the  new  process 
is  N>n+1. 

3.4  Derivation  of  the  Control  Law 

The  minimization  of  the  performance  index  over  the  class  of  possible  pairs 

will  provide  the  solution  to  the  problem  and  the  loss  function  provides 

the  measure  of  achievement  at  each  stage  of  the  process.  To  determine  the  con- 

32  33 

trol  law  we  will  follow  the  framework  of  dynamic  programnlng  ’  and  derive  the 
principle  of  optimality  for  this  case  since  Bellman's  principle  of  optimality 
does  not  apply  directly  when  concerned  with  a  conditional  expectation. 

Consider  the  loss  function. 


or 


N-n+1 


Minimum 


E(J  /D  ) 
'  N-n+l'  n  ' 


N 

T  Minimum  _  ^  pT  „  - 

b‘  1  'i'  ”  J 


(3.9) 


(3.10) 


Taking  the  first  term  of  the  summation  and  writing  it  separately. 


N-n+1 


r  ”  1 

Minimum  _  I -T  „  -  V  -T  _  I 


Writing  the  class  of  pairs  for  which  the  minimization  is  to  be  acconqpllshed,  we 

have 
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Mj^n  llin  . . ,  Min 

=  <P^£^)  (Pj£j)  (  ) 

i=n+l 

(3.12) 

Examlnd.ng  the  first  term  in  the  expression  (3.12),  we  see  that  it  is  the  perfor¬ 
mance  measure  of  system  error  at  time  t  .  The  sequence  of  pulses  that  have  been 

n 

applied  before  time  t^  are  initial  decisions  that  have  already  been  made.  There¬ 
fore,  the  remaining  decisions  must  constitute  an  optimal  policy  with  regard  to  the 
state  resulting  from  the  application  of  the  previous  pulses.  We  may  write  then. 


N 


[i'^Q  Z  +  y 

L  n  n  n  L, 


J 


_  Minimum  v 

^N-n+1  ■  (p  ,€  )  ®L^^n  \  *n^^n-l^ 

n- 1  n- 1 


Min  Min 

*  (p  I  )  (P  ,) 

n  n  n+1  n+1 


■(pX>  '  i 


i=n+l 


Note  that  the  last  term  in  the  expression  is  and  write 


_  Minimum 

N-n+1  "  (P  ,  £  , 

n-1,  n-1 


,  e\  z’^Q  z  /D  ,  +  1  (3.14) 

)  L  n  n  n  n-1  N-nJ 


where  I  =0  and  n  =  l,2,  . .  N  . 

The  introduction  of  the  effect  of  future  errors  is  contained  in  the  derivation 
above  when  the  minimization  terms  are  distributed  throughout  the  expression  and 
the  one  retained  is  (p^  1^*  This  pair  corresponds  to  the  pulse  applied  at 

time  and  the  performance  measure  at  time  t^  is  minimized  by  this  pair. 

Thus,  the  function  of  error  one  sample  ahead  of  the  time  of  application  of  the 
pulse  is  the  future  function  of  error  that  is  minimized.  The  loss  function 
is  the  number  representing  the  achievement  of  this  pulse. 

At  this  point,  there  certainly  exists  the  possibility  of  looking  ahead 
further  in  the  system  to  minimize  the  error  function  perhaps  at  two  sample 
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Instants  ahead.  These  results  may  easily  be  seen  and  we  will  not  carry  through 
the  details  of  that  particular  derivation  here. 

Continuing  with  the  solution  of  the  recurrence  relation  Involving  the  loss 
functions,  we  will  first  let  n=N  and  solve  for  a  single  stage  process.  This 
will  acccnnpllsh  the  first  step  in  the  Iterative  procedure  for  the  final  solution. 
For  n  =  N  equation  (3.14)  is 


,  Minimum  „  .-T-  -  . 

*  ^^nVi/°N-1^ 


(3.15) 


since  I  =0. 
o 


Introducing  the  system  constraint  by  substitution  for  z  from  equation  (3.7), 

N 

,  Minimum  „  ^  -  7  7  v  T. 

^1  “  L^^N-l’^N-l  ^N-l®N-l  ^N-l^N-1^ 

N- 1  N- 1 

•••  ^ViVi  ^  ViVi  ViVi>  /  “n-J 

Performing  the  Indicated  operations  and  expanding, 


Minimum  „r  ^  -  "T  ^  » 

■  (P„  .,€„,)  *N-1^N-1^n‘^N-1*^N-1  ^  ^N-l^^N-l^I^N-l* 

1  K-  1 


N-l 


-T  T 


-T  T 


T  .T 


^  ViVi'*nViVi  ^  Vi^n-iVn-iVi 


+  Q  0  z 

N-l  N-l^N^N-1  N-l 


-T  -T  -  - 

*  *n-iVi^nVi*n-i 


^n-/n-iVn-iVi 

^  Vl] 


(3.17) 


The  first,  third,  seventh  and  last  terms  may  be  rewritten  in  simpler  form 
using  the  following  equivalence. 
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^I^1^N-1*^N^N-1^N-1  ^N-A-l^^N-l^^N-l^N-l 


(3.18) 


A^  Q  0  z  -  z^  0^  Q  A  € 

N-1  N-l^N^N-1  N-1  N-l^N-l^lTK-l  N-1 


Thus,  equation  (3,17)  may  be  written  as 


I,  = 


Minimum  „  T a.  -  ^ 

El  (A  £  +0  z  )Q(A  £  +0  z  ) 

L  N-1  N-1  ^N-1  N-l"^  N-1  N-1  ^N-1  N-1^ 


-T  T 


T 


+  SAQ''  s  +Z0QAS  + 
N-1'^-1^N“N-1  N-1  N- r  N- l^N- l^N- 1  N-1 


(3.19) 


-T  T 

+  s  '  Q  p  z 
N-l'-'N-l  N^N-1  N-1 


-1'  T  ^  T  - 

s  A  Q  A  £  + 

N-1  N-1  N  N-1  N-1 


-T  -T 
+  A.  0 


/  D, 


N-1  N-1  N"N-1  N-1  N-lJ 


If  the  input  to  the  system  is  deterministic  then  A  =0  for  all  n  and  we 

n 

have  only  the  first  term  in  the  expression  (3,19).  If  the  input  is  random  with 
8^  satisfying  the  assumed  properties,  then  taking  the  expectation  with  respect 

to  s  eliminates  all  but  the  first  two  terms, 

n 


Taking  the  expectation  with  respect  to  s  we  then  may  write 


Minimum  a.  - 

1  -^v  ,>  ^  L^^N-l^N-l  ^  Vl^N-1^  ‘^N* 

N-  1  •  N-  i 


.  (A  £  +0  z  )+V  /D  1 

N-1  N-1  ^N-1  N-r  ^o'  N-lJ 


where 


Vo  =  ^s  ^Vi4-1^1^N-iVi%-1^ 


(3.20) 


(3.21) 


In  the  deterministic  case,  v  is  zero  since  A  is  zero  for  all  n.  This 

'  'o  n 

agrees  with  equation  (3.19)  for  this  case. 

To  proceed  further,  the  information  constraint  must  be  considered.  First 
note  that  in  either  the  deterministic  or  random  input  case,  the  state  of  the 
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■ystem  at  any  sample  Instant  Is  not  known  exactly  in  general.  Only  noisy  measure¬ 
ments  of  tha  output  at  the  sample  Instants  are  available  in  the  usual  pase. 

These  measurements  constitute  part  of  the  data  available  for  the  estimation  of 

the  state.  In  particular,  at  time  t  ,  the  data  D  consists  of  any  bit  of 

n'  n 

information  about  the  state  of  the  system,  Including  any  past  record  or  past 
state. 

It  is  known  from  estimation  theory  that  the  best  estimate  of  a  random  variable 

in  the  sense  of  minimizing  a  quadratic  error  criterion  is  the  mean  of  the  a 

26 

posteriori  distribution  which  in  this  case  is  the  expected  value  of  the  state 
given  the  information  or  data  D. 

Using  the  conditional  expectation  then,  we  have 

Vl  =  * 


and  will  call  this  estimate  optimxun  leaving  the  details  of  the  conditional 
expectation  as  an  optimum  estimator  to  Chapter  5. 

Assuming  that  the  optimal  estimate  of  the  state  is  available  then,  when 
applying  the  information  constraint,  equation  (3.20)  reduces  to 


,,  Minimum  \  .-i  r  ,  ^  -*  xT 

^1  '  L^^N-1^N-1  ^N-1  N-1^  S* 

N-1'  W-l 

....(Aj^lCj^l  +  0j,-lVl^  ^  ^o] 


(3.23) 


-♦ 

To  Include  all  cases  of  interest,  we  will  let  z„  ,  represent  the  estimate 

N- 1 

of  the  state  in  the  noisy  case,  and  the  exact  value  of  the  state  if  it  is  known. 

Equation  (3.23)  is  minimized  for  a  best  pair  (p„  ,)•  The  details  of 

N“  1  W"  1 

this  determination  is  left  for  the  general  case.  For  the  present,  assume  that 
a  best  or  optimal  pair  exists.  The  minimum  value  of  the  quadratic  form  is  then 


opt 


(3.24) 


The  loss  function  I^  has  a  value  corresponding  to  this  choice  of  pulse  width 

and  sign. 
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.  E(B^  +  Vo> 


(3.25) 


Continuing  the  step  by  step  solution  let,  n  s  N-1, 


_  IflnlBun  r_  ,-T  - 


H-l  *l] 


(3.26) 


using  the  previously  determined,  we  obtain 


_  Minimum  f-y-T  v  .. 

^2  “  L*^*M-1  ^»-1  ®N-1^M-2^'^  ®1  *  ''^oj  (3*27) 

Equation  (3.27)  is  analogous  to  equation  (3.16)  within  the  constants 
and  Y^.  Hence,  we  sMy  follow  the  same  procedure  and  include  the  (B^  +  * 

obtaining 


'N-1 


^2  *  <P,^2^*N-2^  ^ 

••••^^2  ^N-2  *^2  “n-2^  ^  ''^1  *  "'^o  ■*■  ®l]  (3.28) 


where 


■'^l  *  ®  ^■n-2  ^2  ^N-2  ^2  V2^N-2^ 


(3.29) 


Defining  B^  in  a  manner  similar  to  before 

®2  “  ^V-2  *»-2  ^K-2®H-2^  S-1  2*1^2  *  ^»-2®N-2^ 


opt' 

^N-2»®»-2^ 


The  loss  function  is 


(3.30) 


I2  *  E  (Bj  +  Bg  +  +  Yi) 


(3.31) 


The  quadratic  form  to  be  minimized  in  either  the  deterministic  or  random  case  is 
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Minimize  \  ?  a  “*  /*  ?  a  -*  v"l 

^N-2'^N-2^  L^Sf-2  ^N-2  ^N-2  ®N-2^  ^N-1^V2  ^N-2  *  ^».2  *J»-2^J 


(3.32) 


where 


^N-2  ■  ®^^N-2^N-2^ 


(3.33) 


Suflonarlzing  the  results  for  the  first  two  stages; 


n  a  N 


^  =  B  (Bi  +  Y,) 


(3.34) 


Minimize  [  t  _*  t  ^  a  -*  xl 

<P  ?„  ,)  L^N-1  ^N-l  ^N-1  *N-1^  ^N-1  ^N-1 

IHi,  W-i 


B,  = 


^^N-l  ®N-1  ‘Vl  *N-1^  ^N^Vl  ^If-1  ^  ^fhl  Vl^ 


opt(Pj^l,€^^) 


Y^  a  E  al_^  Vi^N-1^ 


n  a  M>1 


Ig  =  E  (Bj  r  Bg  +  Yo  +  Vj) 


(3.35) 


Minimize  f /»  5  a  "*  xT.x  /i  z  A  -*  xl 

L^^N-2  *N-2  ■*■  ^N-2  ^N-2^  ^N-l^^N-2  S-2  ^N-2  N-2^J 

rr*2  N“2 


®2  “  ^^-2  ^N-2  *^N-2  *N-2^  ‘*N-1^^2  ^N-2  *  ^N-2  ^H-2^ 

b] 


'P‘<^N-2'V2> 


''^l  “  ®  ^®N-2  '^N-2  ^N-1  ^2  ®N-2^Nr2^ 


The  general  term  for  the  nth  point  in  the  N  stage  process  may  simply  be 
written  now  by  induction. 
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N-n 


I  *  E 

N-n+1 


X  (Yi  +  ®i+i^  n=1^2,  ...N 


1=0 


(3.36) 


Th«  quadratic  form  to  be  minimized  Is 


Minimize 

n-l’’n-l 


(P  ..€ 


)  [<VlVl  ^  ♦rlVl'X'Vl*.-!  *  ♦n-lCl*]  «•”> 

for  n  =  1,2, . N 


,  =  (A  ,€  ,  +  <#>  (A  ,£ 

N-n-t-1  n-1  n-1  n-1  n-1  ^n  n-1  n- 


+  <t>  ' 

1  ^  ^n-1 


^*n-l> 


(3.38) 


where 


and 


m’’  Q’  m 
n  n  n 


''^N-n  "  ®  ^^n-l^n-l^^n-l^n-l^n-l^n-l^ 


(3.39) 


The  control  law  will  be  obtained  from  the  minimization  of  the  quadratic 
form  (3.37).  Before  acccnapllshlng  this,  let  us  examine  the  significance  of 
this  particular  form.  Define 


77  =  M  (A  -€  ,  +  0  ,  z  , ) 

'n  n  n-1  n-1  n-1  n-1 


(3.40) 


or 


% "  [<  -“’J 


A  , 

,  0 

■_*  “ 
X  . 

n-1 

2  4. 

^n-1 

n-1 

0 

n-1 

0  9  , 

n-1 

n-1 

(3.41) 


Expanding  (3.41) 


77  =  II*A  +  mV  ,5*  ,  -  mV  ,  (3.42) 

'n  n  n-1  n-1  n  n-1  n-1  n  n-1  n-l 
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t 

I 

I 

I 


Consider  the  last  term  In  (3.42).  Only  terms  In  the  input  process  are  Involved. 
The  input  process  was  defined  by  (2.12). 


~  Ay  - 

W  s9  W  ,+  A  ,8  , 

n  n-1  n-1  n-l  n-1 


r  B  u  w 
n  n  n 


Taking  the  conditional  expectation  with  respect  to  w  we  have 


or,  in  simpler  notation 


%/n-l  ~  *^n-l  *n-l  * 


(3.43) 


since  the  process  input  vector  satisfies  the  original  requirements  of  Chapter  2. 
Also, 

*  r*/  ,*“*«*/  ,  (3.44) 

n  n-1  n/n-1  n  n/n-1 

Combining  these  terms  we  obtain 


.* 


--  ..w  --  aW  "* 

r.  ,»llw.  ,aM<f>  ,w  , 
n/n-1  n  n/n-1  n  n-1  n-1 


(3.45) 


Substituting  this  expression  into  (3.42)  for  rj 


77  =  M*A  ,€  ,  +  ,x*  -  r* ,  , 

'n  n  n-1  n-1  n  n-1  n-1  n/n-1 


(3.46) 


The  remaining  terms  are  involved  with  the  plant  process.  Repeating  these 
equations  for  convenience, 


c  =  M*  X 
n  n  n 


t  B  <|>  ,  X  ,  +  A  € 

n  ’^n-l  n-1  n-1  n-1 


(3.47) 


Taking  the  conditional  expectation  of  the  first  equation  in  (3.47),  we  obtain 

K(-c^/D^^)  .  ^  .  1^  E(;„/D^^>  (3.48) 


I 
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or 


Taking  the  conditional 


“♦  _x  -* 

C  .  ,  =  M  X  .  - 

n/n- 1  n  n/n- 1 


expectation  of  the  second  equation, 


n/n-1  n-1  n-1  n-1  n-1 


Thus, 


c* ,  ,  =  II*  0  ,  i*  ,  +  II*  A  ,€  , 
n/n-1  n  n-1  n-1  n  n-1  n-1 


(3.49) 


Using  the  expressions  (3.45)  and  (3.49)  In  (3.42)  we  see  that. 


n  =  E(c  -r/D  ,)=c,  , 

'n  n  n  n- 1  n/n- 1  n/n- 1 


or 


=  e 


n/n-1 


(3. SO) 


The  function  n  Is  slnply  the  expected  value  of  the  error  at  time  t  given  the 
n  n 

data  at  tine  t 

n- 1 

The  quadratic  fora  to  be  minimized  then  simply  represents  the  measure  of 
system  error  at  one  sample  Instant  ahead  of  the  time  at  which  the  pulse  will  be 
applied.  Obviously,  the  dynamic  programming  approach  here  sinq;>ly  provides  the 
organized  logical  structure  to  follow  In  arriving  at  the  final  result. 

3.5  The  Control  Law 

The  minimization  of  the  quadratic  form  (3.37)  results  In  the  control  law 
for  optlMl  behavior  of  the  system.  Summarizing  the  pertinent  equations, 


vdiere 


Minimize  F  T  "1 

I-’’"  J 

77  ■  M*A  ,  +  11*0*  ,x*  ,  -  r  ,  , 
'n  n  n-1  n-1  n  n-1  n-1  n/n-1 


Let  the  function  H  be  defined  as 
n 


H*  .  -  M*  0*  , 

n  n/n-1  n  ^n-1  n-1 


(3.51) 


(3.52) 


(3.53) 
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This  function  then  takes  Into  account  the  results  of  the  optimum  estimation  of 

the  state  of  the  system  given  the  data  D  , .  The  calculations  and  details  of 
*  n-1 

obtaining  H  are  accomplished  in  Chapter  4.  We  will  assume  temporarily  that 

the  results  of  the  estimation  are  available  and  the  function  H  is  known.  The 

n 

expression  for  using  the  results  of  the  estimator  is  thus 


n  =  M*  A  I  -  H 
'n  n  n-1  n-1  n 


(3.64) 


Using  the  expression  for  we  obtain 


,  f 

n  ^n-1  J 

0 


B  d\  6  -  H* 

n-1  n 


for  i  =  1,2, 


(3.55) 


Define 


for  1  =  1,2,...  .,q 


Equation  (3.54)  may  now  be  written 


(3.56) 


n  =  F(/5  -,t  -t  ,)  e  , 

'n  n-l’n  n-1  n-1 


H 


* 

n 


To  simplify  the  notation,  let 


or 


O  =  the  length  of  the  nth  sampling  Interval 
n 


Then, 


n-1 


)  =  0*(ffjj) 


and  since  the  output  matrix  does  not  depend  on  n 
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Equation  (3.56)  reduces  to 

%  ■  '<*'.-1'  V  ‘n-l  -  <  «•«> 

Instead  of  referring  to  the  nth  sasipllng  Interval,  the  notation  may  be  further 
simplified  by  referring  to  any  sampling  Interval,  and  we  may  write 


n  =  KCP,0)  €  -  H* 


(3.58) 


where 

O  a  the  length  of  the  sampling  Interval 

P  s  the  pulse  width  vector  associated  with  the 
particular  sampling  Interval 

€  s  the  sign  vector  associated  with  the 
particular  sampling  Interval. 


Returning  to  the  quadratic  form  with  this  new  notation,  we  have 


or 


Minimize  rjT 

(P,i)  ** 


-  H*]^Q'[r(P,a)  -  H*] 


(3.59) 


The  condition  to  be  taken  to  Insure  minimization  of  (3.59)  will  be  the  control 
law.  The  control  law  then  must  satisfy 


[F(P,a)  £  -  H*]  (3.60) 

* 

The  function  H  is  known  from  the  results  of  estimation  of  the  states  of 
both  the  plant  and  Input  processes.  Knowing  the  length  of  the  sampling  inter¬ 
val  or  using  the  best  estimate  available,  we  have  sufficient  information  to 
satisfy  condition  (3.60)  by  choice  of  the  pulse  width  and  sign  which  will  be 
optimal  In  the  sense  defined  here. 

It  simply  remains  to  investigate  the  function  F(P,<7)  for  various  types  of 
systems  since  the  estimation  problem  Is  covered  in  Chapter. 4.  The  sign  vector 
C  nay  easily  be  determined  for  the  situation  at  hand  and  this  bears  no  further 


comment 
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For  a  single  variable  control,  the  function  F(P,(T) 
aliq>le  fora 


P 

F<p,ff)  =  II*  J  B  dp 
0 


takes  a  particularly 


(3.61) 


If  the  output  of  the  system  Is  directly  related  to  one  of  the  stite  variables 
the  output  matrix  la  quite  simple  and  fortunately,  this  is  the  usual  case.  In 
particular, 

M*  =  [l  0  0  .  o] 

and  since  the  order  of  the  numerator  Is  usually  less  than  that  of  the  denominator 

for  the  plant  transfer  function,  the  matrix  B  also  contains  several  zeros.  The 

calculation  of  F(P,<7)  la  then  quite  simple  and  only  a  little  more  Involved 

than  the  calculation  of  the  characteristic  matrix  which  may  be  accomplished 

34 

easily  by  means  of  Mason's  signal  flow  graph. 

The  function  F(P,<f)  Is  shown  In  Table  1  for  several  typical  plant  processes. 

Since  P  <  <7  at  all  times,  the  boundary  curve  may  be  considered  to  divide  the  mode 

of  operation  f mom  on-off  control  to  pulse  width  control.  It  Is  apparent  that  In 

some  cases,  the  expression  for  pulse  width  may  be  obtained  In  closed  form  as  a 

* 

function  of  the  estimation  H  and  the  sampling  Interval  <7.  This  expression  will 
satisfy  the  condition  for  optimality  and  constitutes  the  control  law  in  these 

cases. 

TABLE  1 


G(s) 

F(P,<7)  p  <  a 

Boundary  P^ 

1 

-<7  P 
e  (e^  -  1) 

-P 

•  +  1 

1 

■(■+1) 

-<7  0 

P  -  e  -  1) 

P  -  (1  +  e"^  ) 

P(<7  +  1)  -  ^  +  +  1  ) 

P(^  +  1)  +  1  +  e"^ 

p  (e"^-3)  (o“^+  1) 

fiSBtHSM 

2^  4 
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The  technique  which  results  in  optimal  control  can  be  reviewed  easily  using 
the  curves  showing  the  relationship  between  pulse  width,  sampling  length  and  the 
function  F(P,0’).  Following  the  idea  of  using  the  pulse  width  as  a  parameter, 
the  curves  may  be  presented  as  shown  in  Figure  2, 


F(P,a) 


Figure  2.  Typical  Characteristic  Curve 


* 

For  a  known  value  of  the  estimation  function  H  and  the  sampling  length,  the 
optimal  pulse  width  and  sign  are  determined  so  that  the  condition  for  optimality 
is  satisfied.  Repeating  this  for  convenience  we  have 


[p(P,a>e  -  h’ 


(3.60) 


These  "characteristic"  ciu'ves  are  shown  in  Figures  3  to  6  for  several  systems 
including  ones  to  be  used  as  examples  in  Chapter  6. 


3.6  Controller  Mechanization 

The  program  or  mechanization  of  the  procedure  reviewed  in  terms  of  the 
"characteristic"  curves  constitutes  the  design  of  the  optimal  controller.  The 
control  computer  may  be  programmed  to  execute  a  sequence  of  logical  orders  for 
each  sampling  Instant,  determining  the  optimal  pair  (P,C).  A  flow  graph  of  a 
general  scheme  is  shown  in  Figure  7  where  the  Inputs  are  the  length  of  the  . 

-  * 

sampling  Interval  or  its  best  estimate,  and  the  optimal  estimate  of  the  state  z  . 

Let  us  consider  possible  realizations  for  two  simple  classes  of  systems 
with  emphasis  on  the  calculations  to  be  executed  for  each  sampling  Instant. 


I 

I 

I 

I 

1 


Flgur*  3.  Cb«ract«rl8tic  Curve*  for  0(*) 


1 

•■•■1 


i 

I 
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I 

I 

I 

I 


1 

I 
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1 


Figure  6.  Characteristic  Curves  for  0(8) 


8(s+3)(s^  +  2s  +  2) 


Estimator 


Figure  7.  Flow  Graph  of  Controller  Where  F  =  F(P,a) 
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First  consider  s  type  zero  system  with  a  transfer  function 


The  transition. 


G(s) 


s  -f  a 

distribution  and  output  matrices  are; 


A<x-a  B>K  Msl 


The  function  FCP,<7)  is 
T(P,(T) 

In  this  case  r(P,<T)  >  0 
Thus, 


P 

s  li  r  e”^  B  dp  =  e~*  ^  f  (e^ 

•*0  * 

for  0  <  P  <  <T 
€  s  Sign  of  (H*)  as  8gn(H*) 


1) 


(3,62) 


(3.63) 


To  achieve  a  zero  minimum  of  equation  (3.60),  the  pulse  width  is 

P  =  1  Ln[l  +  I  e**'  H*  «]  (3.64) 

e  -4i 

Writing  H  in  terms  of  the  optimal  estirngte  z  we  have, 

H*  »  ^-1  l]  M  0(0)  z*  (3.65) 

The  logarithm  in  equation  (3.64)  must  be  calculated  to  determine  a  pulse  width, 
and  then  checked  to  Insure  the  condition 

0  <  P  <  O 

The  complete  realization  for  the  controller  is  shown  in  Figure  -8.  In  this 
realization,  the  voltages  representing  the  pulse  width  P  and  sign  €  are  multi¬ 
plied  together  and  applied  to  a  hold  circuit  providing  a  staircase  output  which 
is  the  desired  input  signal  to  the  pulse  width  modulator.  These  waveforms  are 
shown  In  Figure  9.  The  complete  block  diagram  of  the  system  is  shown  in  Figure 
10.  If  the  state  is  known  exactly,  no  filter  is  required. 

In  thla  simple  case,  the  value  of  P  has  been  obtained  in  closed  form. 

This  will  not  be  possible  in  general. 


Figure  8.  Controller  Realization  for  G(s) 


o 


Figure  10.  Block  Diagram  of  Complete  System  with  a  First  Order  Plant 
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Consider  a  type  one  eeeond  order  class  of  systems  with  the  transfer  function 


G(s)  = 


K 


s(s-t-a) 

The  transition,  distribution  and  output  matrices  are: 


B  = 


K 

L  J 


M  =  [1  0] 


The  characteristic  matrix  is 


<tKt)  =  e 


At 


1 

a 


-at 


-at 


The  function  F(P,G)  is 


r(P,(n  =  Mp(cr)  J  e'^P  B  dp  «  [w3  -  e"“^(e*^  -  1)]  (3.66) 


where 


0  <  P  <  G 


The  control  law  is  determined  from  equation  (3.60) 


T"”  ■ "’] 


(3.60) 


For  a  zero  minimum, 


%  [.p  -  -  1)]  .  H*e 


(3.67) 


or 


<*)  -  .<'P>  =  V  H*e  -  (.-") 


(3.66) 


Since  F(P,G)  >  o  for  0  <  P  <  (T,  a  >  0  and  K  >  0,  then 


€  s  Sign  of  (H*)  s  sgn(H*) 


(3.69) 
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Bquation  (3.60)  m»y  be  alnlBlzed  by  a  prograa  shown  in  Figure  11.  The 
variable  la  Increaented  and  the  function  tested  at  each  step.  The  prograa  la 
repeated  for  each  saiq)llng  interval. 

Another  possible  realization  nay  be  accoiiq;>llsbed  with  analog  eleaents. 

The  expression 

f<Sp)  «  (aP)  -  (e"*®)  e^*^^ 

is  the  solution  to  a  constant  coefficient  ordinary  differential  equation  which 
Is  easily  prograaaed.  The  value  of  the  function  Is  compared  to  the  remaining 
terms  In  (3.68)  and  If  a  zero  minimum  condition  Is  satisfied,  the  computer  Is 
stopped  with  (sP)  held  on  the  output  of  an  Integrator.  Provision  Is  also  In¬ 
cluded  for  minimization  on  the  boundary  where  P  =  (T.  The  realization  Is 
particularly  simple  since  f(aP)  Is  well  behaved  with 


f  (sP)  >  0 
f(*lP)  <  0 

e 

f(lp)  «  0 


for  0  <  P  <  <f 
for  P**  O 


ellsUlnatlng  the  possibility  of  a  minimum  not  on  the  boundary  for  PaC.  This 
realization  Is  shown  In  Figure  12.  Time  scaling  may  be  Included  to  provide  the 
desired  response  time. 

If  reset  does  not  occur  until  new  data  Is  available,  then  the  holding 
property  of  the  integrators  Is  sufficient  to  eliminate  the  need  for  a  separate 
hold  circuit.  Hence,  a  staircase  wave  Is  fed  directly  to  the  pulse  width  mod¬ 
ulator.  The  block  diagram  for  the  complete  system  Is  shown  In  Figure  13. 

If  exact  measurement  of  the  state  occurs  and  the  Input  to  the  system  Is 
deterministic,  then  no  filter  Is  needed.  This  Is  shown  In  Figure  14. 


Figure  11.  Controller  Realization  for  G(8) 
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Figure  13.  Block  Diagran  for  the  Conplete  System  with  a  Second  Order  Plant 
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Figure  14.  Block  Diagram  for  the  Complete  Syatem  with  No  Additive  Noise  and 
Only  Deterministic  Inputs 
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4.  OPTimaf  PLANT  IDENTIFICATION 


4.1  General 

The  plant  requires  more  state  variables  in  its  description  than  there  are 
measurable  quantities.  Since  the  plant  state  variables  must  be  known  for  com¬ 
plete  knowledge  of  the  plant's  behavior,  the  problem  of  identifying  the  state 
from  the  measurable  outputs  is  apparent.  The  identification  problem  is  equiv¬ 
alent  to  finding  an  optimum  estimator  for  the  state  vector  z  given  knowledge 

.  I  ^ 

of  the  output  vector  Y  for  t  <  t  .  The  initial  derivation  of  the  optimum  linear 

“  26  27  * 

estimator  is  due  to  Kalman  *  and  the  presentation  here  is  simply  a  review  of 
the  derivation  with  emphasis  on  the  interpretation  of  results  applicable  to  this 
problem. 

4.2  The  Conditional  Mean  as  an  Optimum  Estimator 

The  conditional  expectation  following  prop¬ 

erties: 

(1)  It  is  a  linear  function  of  the  output  vector,  requiring 
knowledge  of  only  means  and  covariances. 

(2)  If  z  and  Y  are  gaussian  random  vectors,  it  is  an  optimum 
estimator  for  any  reasonable  performance  index. 

_  _  I 

(3)  If  z  and  Y  are  not  gaussian,  it  is  an  optimum  linear 
estimator. 


For  these  reasons,  the  conditional  expectation  will  be  used  as  the  optimum 
linear  estimator  to  accomplish  the  identification  of  the  state  of  the  system. 


Notatlonal  Definitions; 


“o/o-l  ■ 


_*  - 

e_  *  z_  -  z  =  predicted  estimation  error 


n/n-1 


n 

-a  _ 

e  =  z  -  z  =  estimation  error 

n  “  “ 

cov 


a  ,Y  )  .  eFcJ  -  E(5  ))(Y  -  E(^ 
n'n  Ln  nn  nj 


♦  -* 

P  B  COV  (e  ,e  )  s  the  error  covariance  matrix 
n  n'  n 
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Vn-1  = 

Vl  “  ^^^1*  - ^O 

To  show  that  the  conditional  expectation  Is  an  optimal  linear  estimator^ 
consider  a  simple  case.  Define  the  loss  function  L,  where  Q  is  a  positive, 
seml-definite,  symmetric  matrix. 

(2  -  z  )  Q(z  -  z  )/Y  J  (4.1) 

-*  - 
where  z  Is  an  estimate  of  z. 

By  simple  manipulation,  L  may  be  written 

L  =  [i*-E(i/y’)]'^Q  [z*-E(z/y')]  -  E(i./y' (z'^Qz/y*) 

The  dependence  of  the  estimate  z  is  isolated  In  this  form.  Thus,  the  one  value 

.a 

of  z  iriilch  minimizes  L  Is  clearly 

z*  =  E(z/y’)  (4.2) 

The  estimate  Is  optimum  In  the  sense  of  minimizing  L  and  is  unique  if  Q  is 

.a 

nonslngular.  Thus,  we  speak  of  the  optimum  linear  estimate  z  . 

4.3  The  Optimum  Estimator,  Predictor  and  Filter 

We  will  consider  first  the  problem  of  estimating  the  state  of  the  system 
from  noisy  measurements  of  the  output  assximlng  certain  a  priori  Information. 

The  optimum  estimate  or  mean  of  the  a  posteriori  distribution  Is  calculated  In 
terms  of  the  assumed  a  priori  Information.  The  second  step  is  then  to  show 
how  the  a  priori  Information  may  be  developed  recursively. 

Consider  the  output  relationship 


}  s  set  of  measurements 

at  t  s  t  , 
n-1 


where 


(4.3) 


=  M  z  + 
n  n 


(4.4) 


and  y  Is  the  noisy  measurement  at  t  =  t  available  for  estimation  purposes. 
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Since  the  conditional  expectation  is  linear  with  respect  to  the  output 
vector,  the  form  assumed  for  the  optimal  estimate  is 


-* 

z 

n 


n  n 


z  .  ,  +  b  (y* 

n/n- 1  n  n 


kK-1^ 


(4.5) 


where  is  determined  by  minimizing  the  covariance  of  the  estimation  error. 
The  a  priori  information  assumed  is 


and 


z* ,  ,  ,  E(a  ),  cov(o  ,a  )  and 

n/n-1  ^  n  ^  n^  n 

P**  ,  ,  s  cov(e  ,e  )  s  covariance  of  predicted  estimate 


b  is  a  (m  X  p) 
n 

I  is  an  (m  x  m) 

P  is  a  (p  X  p) 
n 


matrix 
unit  matrix 
matrix 


The  covariance  of  estimation  error  is  to  be  minimized  with  respect  to  the 

elements  of  the  b  matrix.  The  error  in  estimation  is 
n 


e 

n 


z 

n 


(4.6) 


Substituting  (4.4)  and  (45)  into  (4.6), 


_♦  -* 
e  =  z  ,  , 

n  n/n-1 


(M  z  + 
n  n 


'n/n-1^ 


(4.7) 


Since  the 


or 


Computing 


noise  is  additive,  we  may  write 

e*  =  (z*  ..-z  )  +  b  (M  z  +a„)  -  b  (II  z*  +  E(a  )  ) 
n  n/n-1  n  n  n  n  n  n  n  n/n-l  n 


e*  =  (I-b  M  )  (z*  ,  ,  -  z  )  +  b  (S  -  E(o  )) 

n  n  n  n/n-1  n  n  n  n 


the  covariance  of  the  error  in  estimation 


P*  =  cov(o*,e*)  =  (I-b  M  )P  .  (I-b  M  )^  +  b  cov(o  ,o  )b^ 
n  n'  n  n  n  n/n-1  n  n  n  n'  n  n 


(4.8) 


(4.9) 
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The  oovarlane*  matrix  la  S3ramatrlc  and  thus,  minimization  of  the  trace 

n  ' 

with  respect  to  the  elementa  of  b  ia  aufficlent  for  minimizing  the  matrix.  A 

°  ♦ 

neoeaaary  condition  for  minimizing  P  la  found  to  be 

n 


Defining 


F  =  M  P  .  ,  M  +  cov  (a  ,o  ) 

n  n  n/n-1  n  n'  n 


(4.10) 


the  condition  may  be  written 


T  -1 

b  =  P  .  M  F  ^ 
n  n/n-  Inn 


(4.11) 


The  reaulting  mlnlaum  value  for  P  when  using  the  value  of  b  from  (4.10) 

n  n 


la 


*  T 

P  =  P  ,  ,  -  b  F  b 

n  n/n- 1  n  n  n 


(4.12) 


The  value  of  P^  la  the  minimum  covariance  of  estimation  error  attained  by  the 
optimiui  linear  estimator. 

Sumamurizing  the  filter  equations: 


-  (I-b  M  )  \  ^ 

n  n  n  n/iw  i  n  n  n  n 


(4.13) 


where 


F  s  M  P  ,  ,  11  +  cov  (a  ,o  ) 

n  n  n/n-1  n  n^  n 


T  -1 

b  =  P  .  ,  M  F 
n  n/n-1  n  n 


and  the  minimum  error  covariance  for  this  estimate  is 

*  T 

P  *  P  .  ,  -  b  F  b 

n  n/n- 1  n  n  n 


The  filter  equations  in  (4.13)  depend  on  the  assumed  a  priori  information 


-* 


*n/m-l  ^n/n-1'  inforswition  will  be  developed  recursively.  Consider 

the  process 


50 


z  =6  z  ,  +  q 
n  1  n- 1 


n-1 


Azzunlng  P  ,  and  z  ,  are  known,  P  ,  ,  and  z  .  ,  must  be  calculated, 
^  n-1  n-1  *  n/n-1  n/n-1 

the  conditional  expectation  of  (4.14) 

n  ft*  1  11“  X  o*  1  n*  1  n- 1  n*  l 


or 


-*  A  -*  “* 

z  .  ,  =  9  ,  z  ,  +  q  , 
n/n-1  n-1  n-1  n-1 


Calculating  the  errors  in  this  estimate,  we  have 


*n  “  ^*n/n-l  ”  *n^  “  *^n-l  ®nrl  ^**n-l  ^n-1^ 


The  covariance  in  this  estimate  is 


V-i  ■  *  *  [Cl  -  5„.i><Vi  -  Vi’T 

Thus,  the  a  priori  information  is  determined  from  (4.15)  and  (4.17). 
The  complete  estimator  may  now  be  summarized  for  the  process 

z  =9  ,  z  ,  +  q  , 
n  ^n-1  n-1  n-1 


y '  -  ” 

^  =  M  z  +  o 

n  n  n  n 


Given  the  estimate  of  the  state  at  t  s  t  ,  and  the  covariance  of  this 

z  ,  and  P  , 
n-1  n-1 


-* 

z 


-* 


and 


n/n-1  ~  *^n-l  *n-l  ^  **0-1 


♦  T  ••  ap 

P  /  -.P  ^  1+  cov(q  ,,q  ,) 

n/n-1  ^n-1  n-1  ^n-1  'n-l'^n-l 


The  filter  is 


z*  -  (I-b  M)z*.  ,-by’-b  E(o  ) 
n  n  n  n/n-1  n'^n  n  n 


(4.14) 

Taking 

(4.15) 

(4.16) 

(4.17) 


estimate, 

(4.18) 
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I 


Figure  IS.  The  C^tlmal  Filter 
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5.  THE  OPTIMAL  CONTROL  LAW  WITH  IDENTIFICATION 


5.1  General 

The  results  of  Chapters  3  and  4  are  now  combined  in  summary  to  achieve  the 
complete  optimization  technique.  The  optimal  estimates,  assumed  to  exist  in 
Chapter  3,  are  obtained  by  the  Kalman  filter  in  general  form.  That  is,  esti¬ 
mation  for  both  the  input  and  plant  processes  are  combined  into  a  single  effort 
idilch  reduces  to  appropriate  conditions  when  specific  conditions  exist  (e.g.  deter¬ 
ministic  inputs,  etc.) 

The  condition  for  optimality  utilizing  these  estimates  is  shown  in  the  flow 
graphs  for  two  alternative  optimization  procedures. 

5.2  Summary  of  Equations  and  Optimization  Scheme 
The  condition  for  optimal  control  is 


where 


(P 


Minimize 

n-l'^n-1 


) 


>Vi>Vi 


(PH 

F(/5,ff)  =  M*  <tP(<J)  B  dX. 


i  —1 , 2 , . .  .  q 


and 


*  -* 

H  =  r  ,  , 

n  n/n-1 


M  9  , 

n  n-1 


*n/n-l 


(3.60) 


(3.61) 


(3.53) 


represents  the  results  of  estimation  and  has  been  previously  assumed  to 
exist. 


♦ 

Investigating  the  significance  of  H^,  first  consider  the  plant  process 
defined  in  (2.10) 


X  =  9*  X  ,  +  A  € 
n  n-1  n-1  n-1  n-1 


c  =  M  X 
n  n  n 


(2.10) 
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Assuming  thst  no  pulse  is  spplled  st  t  »  t  (A  .  s  o).  snd  taking  the  con- 

n  n—  1  ' 

dltlonsl  expectation  of  the  first  equation  In  (2.10),  we  get 


Multiplying  by  M^  , 


*n/n-l 


M*  x*.  ,  =  M*  ,  X* 
n  n/n-1  n  ^n-1  n-l 


(5.1) 


This  tern  then  la  the  estimate  of  the  output  at  t  s  t^  when  no  pulse  Is  applied 

at  t  »  t  ,  .  Since  H*  Is  the  difference  between  this  term  and  the  expected  value 
n-l  n 

of  the  reference  Input  ,  it  may  be  Interpreted  simply  as  the  estimate  of  the 

future  or  extrapolated  incremental  error  at  t  =  t^  when  no  pulse  is  applied  at 

t  *  t  , .  The  pulse  width  and  sign  detexmilned  from  the  control  law  are  chosen 
n-l 

to  minimize  the  Incremental  error  that  would  occur  at  the  next  sampling  instant 
If  the  pulse  were  not  applied.  A  pulse  exists  then  for  the  sole  purpose  of 
minimizing  the  effect  of  the  nonexistence  of  that  pulse.  This  is  shown  in 
Figure  16. 

Since  the  estimation  Involves  both  the  input  and  plant  processes  In  the 

general  case,  a  compact  form  using  the  stacked  state  vector  z  will  be  used. 

•  -*  “ 

Writing  the  function  H  In  terms  of  z  , ,  we  have 

n  n-l' 


or 


H*  =  -m’'  0*  ,  X* 

n  n  n-l  n-l  n  n-l  n-l 


H  =  [-1  1]  M  <#> 

n  n  n*i 


*n-l 


The  system  constraint  Is 


-<t>. 


n-l  n-l 


n-l 


A  ,  €  , 

n-l  n-l 


y 


n 


M 

n 


z 

n 


(2.15) 


The  Information  constraint  Is  Included  by  considering  measurements  of  the  out¬ 
put  corrupted  by  additive  noise. 


M  z 

n  n 


+  a 


(5.2) 
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Figure  16.  Uae  of  Pulse  to  Cancel  Future  Error 


The  set  of  these  ■eeeuresente  constitute  the  data  to  be  used  for  estlnatlon 
purposes  at  a  particular  sample  time. 


°n-l  "{^n-1'  . ^o} 

The  filter  equations  are  summarized  as  follows: 

Begin  at  tlsw  t  ■  t  and  assuaw  the  knowledge  of  the  optimal  estimate  of  the 

state  z  and  the  covariance  of  the  estimation  error  P 

n-1  n-1 

The  predicted  estimate  of  the  state  at  t  s  t  is 


*n/n-l 


-  (p  ,  z  ,  +  A  , 
n-1  n-1  n-1 


‘n-l 


(5.4) 


The  covariance  In  this  estimate  Is 


**n/n-l  *  *^n-l  **n-l  ^n-1  ^n-1  ^n-1 


(5.5) 


Using  the  measurement  y*  at  t  «  t  .  the  filtered  estimate  Is 

n  n 


z  «  (I-b  ll)z*.  ,“b  y'+b  E(a  ) 
n  n  n  n/n-1  n  'n  n  n 


(5.6) 


where 


and 


F  =M  P  ,  +  cov  (a  ,o  ) 

n  n  n/n-1  n  n^  n 


T  -1 

b  =  P  .  ,  M  F 
n  n/n-  Inn 


(5.7) 


(5.8) 


The  covariance  of  the  filtered  estimate  is 


*  T 

P  =P,  ,-b  F  b 

n  n/n-1  n  n  n 


(5.9) 


In  writing  equations  (5.4)  to  (5.9),  the  following  definitions  have  been 


made: 


n/n-1 


n/n-1 


n/n-l 
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where 


P  Is  a 

P  is  a 

P*  is  a 

F  is  a 
n 

F  is  a 
n 

F  is  a 
n 

b  is  a 
n 

b*  is  a 
n 

b  is  a 
n 

a  is  a 
n 

a  is  a 
D 

~w  ^ 

a  is  a 
n 


(m-fv  X  m-f-v)  covariance  matrix 

(m  X  m)  covariance  matrix  for  the  plant  state 
vector 

(v  X  v)  covariance  matrix  for  the  input  process 
state  vector 

(2p  X  2p)  matrix 

(p  X  p)  matrix  for  the  plant  process  filter 

(p  X  p)  matrix  for  the  input  process  filter 
(m+v  X  2p)  matrix 

(m  X  p)  matrix  for  the  plant  process  filter 

(v  X  p)  matrix  for  the  input  process  filter 
(m+v  X  1)  noise  vector 

(m  X  1)  noise  vector  for  the  plant  process 
(v  X  1)  noise  vector  for  the  input  process 


For  the  case  of  deterministic  inputs  or  exact  measurement  of  the  state, 

the  general  filter  equations  reduce  to  the  appropriate  conditions.  If,  for 

exsmple,  the  problem  is  coiiq)letely  deterministic,  then  ^  ^ 

function  H  represents  the  exact  incremental  error  Involved, 
n 

The  block  diagram  for  the  complete  optimization  procedure  is  shown  in 
Figure  17. 
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ctor  and  Filter 
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5.3  Alternate  Scheme 

For  the  program  shown,  there  is  a  limitation  due  to  the  finite  time  required 
for  the  calculations  Involved.  The  pulse  applied  at  time  t  =  t^  must  be  cal¬ 
culated  using  the  data  obtained  at  that  same  time.  The  optimal  estimate  must  be 
found  and  the  procedure  for  determining  the  optimum  pulse  width  must  be  executed 
before  the  pulse  may  actually  be  applied.  The  time  required  for  these  calcula¬ 
tions  delays  the  application  of  the  pulse  sufficiently  in  some  cases.  If  this 
limitation  is  too  severe  due  to  high  sensitivity  of  the  output  with  respect  to 
pulse  width,  or  the  system  is  operated  with  a  fast  sampling  rate  compared  to  the^ 
time  of  calculation,  then  an  alternate  procedure  must  be  developed. 

The  alternate  procedure  is  quite  simple  in  that  predicted  estimates  rather 
than  filtered  estimates  are  used  where  the  prediction  and  other  calculations  are 
executed  throughout  the  sampling  Interval.  That  is,  at  time  t  =  t^,  a  measure¬ 
ment  of  the  output  is  available,  but  will  be  used  for  predicting  the  state  at 

♦ 

t  =  t  , .  The  function  H  ,  is  determined  from  this  predicted  value  and  the 
optimal  pulse  width  and  sign  are  then  calculated.  These  operations  are  per¬ 
formed  during  the  sampling  interval  and  the  pulse  to  be  applied  at  the  next 
sampling  Instant  is  ready.  This  alternate  procedure  limits  the  calculation 
time  to  the  length  of  the  sampling  interval,  but  the  obvious  extension  of  pre¬ 
dicting  several  samples  ahead  may  be  used  in  cases  of  this  nature. 

The  flow  graph  for  the  alternate  scheme  is  shown  in  Figure  18. 
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Figure  18.  Block  Diagram  of  Alternate  Scheme 
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6.  EXAJIPLES 


6.1  General 

This  chapter  is  Intended  to  Illustrate  the  optimization  technique  described 
in  the  previous  chapters.  It  is  hoped  that  the  validity  and  usefulness  of  the 
theory  will  be  demonstrated  at  the  same  time.  Numerical  results  have  been  ob¬ 
tained  through  simulation  of  the  entire  system  on  a  high-speed  digital  c<mnputer. 

The  computer  Involved  in  this  simulation  was  the  Control  Data  Corporation 

1604.  The  entire  program  was  written  using  the  FGRTOAN  compiler  language.  The 
35 

extensively  tested  normal  deviates  from  the  table  A  MILLION  RANDOM  DIGITS 

36 

WITH  100,000  NORMAL  DEVIATES"  were  transcribed  in  part  on  magnetic  tape  and 
used  as  the  source  of  additive  noise  in  all  cases.  The  program  included  the 
choice  of  mean  and  covariance  for  these  normal  deviates.  A  random  number  gen¬ 
erator,  which  was  programmed  in  machine  language,  together  with  von  Neumann's 
37 

rejection  technique  for  sampling  from  various  other  distributions  was  also 
available. 

The  optimal  response  is  compared  to  a  normal  response  in  each  case  where 
normal  pulse  width  control  refers  to  a  standard  procedure  used  to  determine  the 
pulse  width  and  sign  at  each  sampling  instant.  This  standard  procedure  is  some¬ 
times  called  "lead"  pulse  width  modulation  where  the  width  and  sign  of  each  pulse 
are  directly  related  to  the  magnitude  and  sign  of  the  actuating  signal  at  the 
sampling  Instant. 


€  =  sign  of  (e  ) 

n  n 

The  parameter  "a"  was  taken  as  unity  for  convenience. 

The  block  diagram  of  the  system  under  consideration  for  each  example  is 


shown  in  Figure  19. 
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Noise 
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The  discrete  form  is 


where 


X  ,  =  ^  X  +  A  £ 
n+1  n  n  n  n 


c  =  M  X 
n  n  n 


At 

9  (t)  =  e 


1  1  -  e 


0  e 


-t 


r 


-At 


Y  =  e  B  dt  = 

'o 


P  -  (e^  -1) 


(e^  -1)  J 


A(P,a)  =  4>*(a)  y  = 


rp.  -1)1 


-1) 


The  step  responses  for  plant  gains  of  4.0  and  5.0  are  shown  in  Figures  20 
and  21  for  both  exact  and  noisy  measurements  of  the  state.  The  additive  noise 
was  assumed  normally  distributed  with  zero  mean  and  unity  covariance.  The 
sampling  rate  was  assumed  constant  at  0.1  seconds. 

The  normal  pulse  width  response  was  found  to  be  unstable  for  any  gain 
greater  than  5.0.  The  optimum  system  was  found  to  be  stable  for  gains  much 
higher  than  5.0  for  both  step  and  ramp  Inputs.  The  response  for  a  gain  of  10.0 
is  shown  in  Figure  22. 

The  ramp  responses  for  these  gains  are  shown  in  Figure  23. 

The  results  show  a  marked  Improvement  in  performance  for  these  Inputs. 

For  the  step  response^  this  is  attributed  to  the  small  pulse  applied  before 
the  desired  value  is  reached  allowing  the  system  to  coast  for  the  remainder 
of  the  sampling  interval  and  then  settle  rapidly  thereafter.  This  of  course 
is  due  entirely  to  the  minimization  of  the  future  errors. 

The  flow  graph  for  a  digital  controller  is  shown  in  Figure  24.  The 
analog  realization  is  shown  in  Figure  25. 
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Figure  20.  Step  Reeponee  of  Second  Order  System  0(8)  s 
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s(s-«-l) 


Figure  21.  Step  Response  of  Second  Order  System  G(s)  = 


c(t) 
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f 


Figure  22.  Step  Response  of  Second  Order  System  G(s)  = 
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Figure  23,  Ranp  Reeponse  of  Second  Order  System  0(8)  = 
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Figure  24. 


Flow  Graph  of  Controller  and  Predictor 
Program  for  Second  Order  System 


G(b) 


_  _K _ 

s(s+l) 
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6.3  A  Third  Order  System 

A  third  order  system  with  complex  roots  is  investigated  in  this  section. 
The  transfer  function  of  the  plant  is 

G(8)  - - - - ^ -  (6.3) 

s(s  +  2b  +  2) 

The  block  diagram  of  the  system  is  shown  in  Figure  19.  Preliminary  calcu¬ 
lations  lead  to  the  following  matrices: 


"o  1  0 

o' 

A  = 

0  0  1 

B  = 

0 

0-2  -2_ 

K 

«*<t)  =  e""^ 


1  1  -  e”^  Co8(t)  I  -  ^  e~^  Sin(t+45°) 

0  <2  e"*  Sln(t+45°)  e"^  Sin(t) 

0  -2  e“^  Sin{t)  -  42  e’^  Sin(t-45®) 


Y 


B  dt 


e  -  i  ^  co.<P)  .  i 

-  -i  .  ^Cos(Pt45°) 

^  Sln(P) 


A(P,a)  =  Y 


A  plant  gain  of  2.0  and  a  constant  sampling  rate  of  1.20  seconds  was  used 
for  this  example.  The  unit  step  response  is  shown  for  exact  and  noisy  measure¬ 
ments  of  the  state  in  Figure  26. 

Since  a  nonlinear  system  is  under  Investigation,  the  unit  step  response 
has  very  little  significance  in  terms  of  describing  the  system  performance  to 
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Flgur*  26.  Step  Reiponse  of  Third  Order  Syetem  G(s)  - - = - 

s(s  +  2s  -I-  2) 
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dlsplftMMat  inputa.  For  this  roMoa,  •  faally  of  stop  responaea  la  shown  In 
Figwo  27  for  tbo  opt  Inal  nodo.  The  eorreapondlng  nomal  responses  are  shown 
In  Fiftire  28.  A  direct  comparison  is  shown  in  Figure  29. 

For  large  step  li4>uts  the  system  approaches  the  behavior  of  the  normal 
system  since  the  on>off  mode  of  operation  is  in  effect  in  both  cases  until  -Just 
before  settling  time.  Also,  since  the  system  response  time  is  very  slow  for 
large  inputs,  little  Information  will  be  gained  by  considering  these  large  dis¬ 
placement  inputa. 

For  small  step  inputs,  the  behavior  depends  a  great  deal  on  the  sampling 
rate  and  begins  to  exhibit  rather  sporadic  results  for  large  sampling  rates  com¬ 
pared  to  the  system  response  time.  The  criterion  for  optimality  is  satisfied 
In  all  eases,  but  it  becosws  doubtful  whether  or  not  the  criterion  has  any 
Maning  in  relation  to  what  we  night  consider  good  response  with  respect  to  our 
aesthetic  criterion  for  optimal  behavior.  A  good  example  of  this  is  seen  for 
the  0.70  step  response  in  the  optimal  mode.  The  system  eventually  settles  and 
satisfies  the  criterion  established,  but  our  Judgment  dictates  that  the  normal 
response  In  this  case  is  better  with  respect  to  our  aesthetic  optimality  cri¬ 
terion.  Thus,  the  optimal  node  leads  to  undesirable  results  for  some  ranges  of 
operation.  The  pole-zero  location  In  relation  to  the  magnitude  of  input  infor¬ 
mation  for  various  sampling  rates  is  seen  to  be  an  Important  consideration 
when  using  this  optimization  technique  since  there  are  desirable  and  undesirable 
ranges  of  operation. 

No  attempt  was  made  to  Investigate  thoroughly  the  change  of  stability 
boundaries  with  respect  to  gain  and  saBqi>llng  rate  but  there  seems  to  be  an 
extremely  interesting  effect  involved.  For  a  gain  of  3.0  and  a  unit  sampling 
rate,  the  system  is  stable  in  the  optimal  mode  but  unstable  in  the  normal 
mode.  On  the  other  hand,  for  a  gain  of  2.0  and  a  unit  sampling  rate,  the  re¬ 
verse  is  true.  The  explanation  for  this  may  be  tied  in  with  the  fact  that 
these  points  lie  very  near  the  stability  boundary. 

For  a  ramp  liq>ut,  the  system  exhibited  a  constant  steady  state  positional 
error  of  2.20  units  in  the  normal  mode  when  the  state  was  known  expctly.  In 
the  optimal  mode  for  exact  measurement  of  the  state,  the  steady  state  positional 
error  was  Improved  to  1.0  units..  In  the' optimal  mode  when  the  information  about 
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Figure  27.  Family  of  Optimal  Step  Responses  for  Third  Order  System 
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s(s  +  2b  +  2) 
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Figure  28. 


Family  of  Step  Responses  for  Third  Order  System 
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Figure  29.  Step  Response  of  Third  Order  System  G(s) 
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th«  state  was  conatralned,  the  ateady  atate  error  was  found  to  be  a  linear 
function  of  tlae.  These  results  are  shown  In  Figure  30.  The  steady  state 
error  series  for  each  case  la  listed  as  follows; 

Normal  mode;  exact  measurement  of  the  state 


E  (t)  >  2.20 
ss 

Optimal  mode;  exact  measurement  of  the  state 


E,_(t)  =  1.00 

8B 

Optimal  mode;  noisy  measurement  of  the  state 


E  (t)  =  0.346  t  -  6.5 
ss 

6.4  K  Fourth  Order  System  with  Distinct  Roots 

A  fourth  order  system  with  a  zero  Is  Investigated  In  this  section.  The 
transfer  function  of  the  plant  Is 
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The  block  diagram  of  the  system  Is  again  In  Figure  19.  Preliminary  calculations 
of  the  necessary  matrices  lead  to  the  following  results; 
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The  charecterletlc  matrix  for  higher  order  systems  is  somewhat  involved  but 

34 

may  be  calculated  easily  using  Mason's  gain  formula  as  presented  by  B.C.  Kuo  . 

A  faisily  of  step  responses  is  shown  in  Figure  31  for  a  gain  of  6.0  and  a 
unit  saaplirg  rate.  Again  there  is  a  range  of  inputs  which  lead  to  results 
which  may  be  undesirable.  In  particular, the  input  step  of  0.3  units  begins 
following  the  0.4  unit  input  response  until  about  2.5  seconds  resulting  in  a 
large  initial  overshoot.  This  result  of  course  depends  on  the  sampling  rate 
chosen  and  is  expected  in  this  case  since  the  rate  is  quite  low  compared  to  the 
settling  time  for  this  lower  input  level.  In  general,  we  would  expect  unusual 
results  when  the  sampling  rate  is  larger  than  the  system  response  or  settling 
time. 

In  order  to  compare  the  normal  and  optimal  responses,  a  quality  factor 
which  seems  to  be  reasonable  is  the  5^  settling  time  for  the  two  systems. 

Figure  32  shows  the  settling  time  versus  the  magnitude  of  displacement  input. 

For  large  Inputs  the  on-off  mode  is  predominant  and  the  two  curves  approach  the 
same  asymptote  as  shown  in  the  figure.  For  input  magnitudes  greater  than  40.0, 
the  settling  tiM  curves  for  both  systems  are  within  5^  of  the  asymptote. 

Investigating  the  behavior  of  the  two  systems  further,  another  quality 
factor  examined  was  the  first  overshoot  versus  input  magnitude.  This  curve  is 
shown  in  Figure  33  and  we  see  Immediately  that  lower  level  inputs  exhibit  rather 
sporadic  first  overshoots  due  primarily  to  the  large  sampling  rate.  Note,  how¬ 
ever,  that  in  this  case  even  though  the  first  overshoot  is  larger  than  that  for 
the  normal  system  for  an  input  of  0.3  units,  the  settling  time  is  still  about  60^ 
lower  than  the  normal  response. 

Since  the  sampling  rate  is  fixed  at  unity,  it  is  not  to  our  advantage  here 
to  consider  step  responses  for  input  signal  levels  less  than  0.1  units.  A  family 
of  responses  for  various  other  sampling  rates  and  input  levels  would  be  in  order 
however,  if  the  technique  were  to  be  applied  to  existing  systems. 

6.5  A  Fourth  Order  System  with  Complex  Roots 

A  fourth  order  system  with  complex  roots  is  investigated  in  this  section. 

The  transfer  function  of  the  plant  is 
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Figure  31.  Family  of  Optimal  Step  Responaee  for  Fourth 
Order  System  with  Distinct  Roots 
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Th«  Initial  calculations  lead  to  the  following  matrices; 
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The  elements  of  the  characteristic  matrix  0(t)  =  e  are 
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<#>^2(t)  =  e”^  Sin(t-45®)  -  e“^  Sln(t) 

*^43^^^  *  ^  *'^  Sin(t-45°)  -  ^  e"^  Sin(t) 

=  I  Sin(t-45°)  -  |  e"^  Sin(t) 

I  *  ?  -  45<**’  -»  -  ^  -  'f  c»*<P*«°) 

~  io  ■*■  Ts^®^  “  3  ®^  Sin(p)  +  -^  Cos(p+45°) 

-  i  -  -^(e^  -1)  +  -|  SinCP)  +  ^  Co8CP+45°) 

SO  5  0 

+  -1(6*^  -1)  -  ^  Sin(p)  -  Co8(p+45°) 

0  5  3  O 

The  atep  response  curves  for  a  gain  of  5.0  and  a  sampling  rate  of  1,40 
seconds  are  shown  in  Figure  34.  The  input  level  in  this  case  is  2.0  units.  In 
Figure  35  the  response  is  shown  for  an  input  signal  level  of  0.8  units. 

6.6  Comments 

From  the  several  examples  presented  here,  we  see  that  the  system  performance 
is  greatly  Improved  for  certain  ranges  of  gains,  sampling  rates  and  input  signal 
levels  when  optimized  using  the  technique  presented  in  the  previous  chapters. 
There  are,  of  course,  ranges  where  the  response  satisfies  the  criterion  estab¬ 
lished  but  where  it  does  not  satisfy  some  aesthetic  criterion  for  optimal  behav¬ 
ior.  In  particular,  for  the  families  of  curves  shown,  the  sampling  rate  was 
assumed  constant  and  for  low  level  signal  inputs,  the  response  time  of  the  sys¬ 
tem  is  smaller  than  the  sampling  rate  leading  to  results  that  may  be  classified 
as  undesirable.  The  results  demonstrate,  however,  the  ability  of  the  system  to 
achieve  optimal  performance  with  respect  to  the  established  criterion  in  all 
casds  and  excellent  performance  not  otherwise  possible  in  many  cases. 
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Figure  34.  Step  Response  of  Fourth  Order  System  with  Conq;>lex  Roots 
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Figure  35.  Step  Responee  of  Fourth  Order  System  with  Complex  Roots 
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7.  SUMMARY^  FURTHER  PROBLEMS  AND  CONCLUSIONS 
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7.1  Summary 

A  method  for  the  design  of  a  controller  to  optimize  a  class  of  pulse  width 
modulated  systems  has  been  presented.  The  process  to  be  controlled  is  linear, 
time- Invariant,  of  arbitrary  order  and  excited  by  a  sequence  of  pulses  generated 
from  information  available  at  arbitrary  sampling  instants.  State  variable  no¬ 
tation  has  been  used  for  description  of  the  plant  and  input  processes.  Estimation 
of  the  state  at  the  sampling  instants  is  provided  by  linear  estimation  techniques, 
accomplishing  the  identification  problem.  The  information  obtained  from  the 
filter  is  then  used  in  the  contsoller  to  develop  a  predicted  estimate  of  the 
future  system  error  which  in  turn  is  used  to  develop  the  optimum  pulse  width  and 
sign  to  minimize  a  measure  of  the  predicted  future  system  error  at  each  sampling 
instant.  The  criterion  established,  providing  the  definition  of  optimality,  is 
based  on  the  minimization  of  the  conditional  expectation  of  a  sum-squared  perfor¬ 
mance  index.  It  was  shown  that  the  overall  system  is  optimum  in  this  sense. 

Two  alternate  schemes  for  determination  of  the  pulse  width  and  sign  were  pre¬ 
sented  allowing  flexibility  in  controller  design  to  include  those  systems  re¬ 
quiring  calculation  time  in  the  controller  comparable  to  the  length  of  the 
sampling  interval. 

The  construction  of  an  optimal  strategy  has  been  presented  in  the  lltera- 
based  on  the  determination  of  a  finite  canonical  sequence 
associated  with  each  initial  state  which  specifies  optimal  control  over  the  en¬ 
tire  transient  process  during  which  the  state  is  taken  to  the  origin.  Con¬ 
struction  of  the  canonical  sequence  associated  with  any  arbitrary  initial  state 
is  determined  from  the  observed  value  of  the  state  by  a  reverse  time  mapping. 

The  phase  plane  is  divided  into  two  regions  corresponding  to  pulse  width  and 
relay  control  in  each  of  these  cases.  The  approach  is  limited  to  plants  of 
second  order  due  to  the  conceptual  difficulty  of  determining  the  optimal  con¬ 
trol  regions  in  higher  dimensional  state  spaces. 

The  technique  presented  in  this  report  is  based  on  the  philosophy  of 
minimizing  future  system  errors  at  each  sampling  instant  as  opposed  to  the  usual 
reverse  time  mapping.  The  future  weighted  errors  are  mli]^mlzed  sequentially 
and  the  optimal  strategy  is  constructed  at  each  step  removing  the  previous 
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restriction  to  second  order  systems.  Input  information  is  also  quite  general  in 
that  random  and  deterministic  phenomena  are  included  as  opposed  to  the  previously 
presented  techniques  which  are  restricted  to  simple  deterministic  inputs.  The 
usual  condition  in  practice  where  information  about  the  state  is  constrained  is 
also  included.  Sampling  may  occur  at  arbitrary  times  and  is  not  limited  to  a  con¬ 
stant  rate.  Finally,  the  application  of  this  technique  is  no  more  difficult  for 
a  system  of  arbitrary  order  than  for  one  of  second  order.  The  higher  order  system 
requires  more  computation  In  the  controller  but  if  this  restriction  is  severe  an 
alternate  technique  has  been  presented  utilizing  the  length  of  the  sampling  Inter¬ 
val  for  computational  purposes. 

Several  systems  were  simulated  on  a  digital  computer  using  the  methods  pre¬ 
sented  in  this  paper.  The  results  Indicate  superior  performance  of  the  system  for 
ranges  of  input  signal  levels  with  respect  to  the  length  of  the  sampling  Interval 
and  the  gain  of  the  plant  process.  It  was  found  that  Input  displacement  levels  of 
small  magnitude  corresponded  to  response  time  of  the  system  which  was  smaller  than 
the  sampling  interval  leading  to  what  might  be  called  undesirable  results  even 
though  the  criterion  for  optimality  was  satisfied.  Also,  for  input  displacement 
levels  of  large  magnitude,  the  system  was  found  to  be  predominantly  in  the  relay 
mode  and  hence,  pulse  width  control  effort  had  very  little  effect. 

7.2  Further  Problems 

The  concept  of  minimizing  future  errors  has  led  to  the  technique  presented 
in  the  preceding  work.  The  future  error  at  only  one  sampling  instant  ahead  was 
considered  with  only  a  passing  mention  of  the  control  law  which  would  be  appli¬ 
cable  if  future  errors  several  sample  Instants  ahead  were  considered.  The 
extension  of  this  technique  to  include  investigation  of  the  predicted  error  at 
perhaps  two  sample  Instants  ahead  together  with  the  information  about  error  only 
one  sample  ahead  and  a  subsequent  decision  process  based  on  the  desired  perfor¬ 
mance,  might  prove  worthwhile.  Various  approaches  utilizing  the  predicted 
errors  may  provide  the  insight  necessary  to  achieve  excellent  performance  for 
systems  other  than  those  with  pulse  width  control  signals. 

No  investigation  into  the  stability  problem  was  made  in  this  work.  Several 
interesting  results  were  noted  however,  when  the  controller  was  Introduced  into 
the  system.  The  stability  boundary  determined  as  a  function  of  sampling  time 
and  plant  gain  for  two  of  the  example  systems  changed  appreciably.  It  was  found 
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that  a  second  order  system  was  stabilized  in  the  optimal  mode  for  both  step 
and  ramp  inputs  for  gains  twice  as  large  as  the  maximum  gain  allowable  for  sta¬ 
bility  in  the  normal  mode.  A  third  order  system  was  stabilized  in  one  case  by 
allowing  a  larger  gain  than  in  the  normal  mode,  but  on  the  other  hand,  the 
reverse  was  found  true  for  a  different  sampling  rate.  Since  the  system  is  non¬ 
linear,  the  Interesting  stability  problem  would  be  extremely  difficult  to  solve 
but  would  certainly  be  a  major  contribution. 

7.3  Conclusions 

The  techniques  presented  here  are  applicable  to  plants  excited  by  a  pulse 
width  modulated  signal.  The  results  seem  to  indicate  marked  Improvement  in 
system  performance  at  only  the  expense  of  additional  computation  which  may  be 
accomplished  between  the  sampling  intervals.  It  must  be  concluded,  however, 
that  the  science  of  optimization  is  still  in  the  Infancy  stage  and  the  use  of 
predicted  future  system  errors  to  achieve  optimal  control  is  but  only  an  attempt 
which,  hopefully,  will  help  contribute  to  the  maturity  of  optimization  science. 
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